When an image at a first computing terminal is changed, data can be generated which can subsequently be used to form the changed image. Rather than generating data representing the whole changed image, it can be beneficial to generate data representing the changes that have been made to the image, wherein the original image and the data representing the changes made thereto can be used together to form the changed image.
It can be particularly useful to generate data representing the changes to an image when the changes to the image are to be transmitted from a first terminal to a second terminal. There are often bandwidth constraints in transmitting data between terminals, so it can be beneficial to reduce the amount of data that is transmitted between the terminals. Therefore, transmitting data representing the changes to an image, rather than data representing the whole changed image can be beneficial since less data is required to be transmitted. In this way, updates to an image at a first terminal can be transmitted to a second terminal.
One example in which it is useful to transmit changes made to an image from a first terminal to a second terminal is when implementing screen sharing. Screen sharing is a useful technique for communication between two terminals. Images displayed on a first screen at a first terminal (or “sharer” terminal) can be transmitted to a second terminal (or “viewer” terminal) and displayed on a second screen at the second terminal. As an example, screen sharing can be particularly useful when a first user at the first terminal (the “sharer”) is trying to explain what they are seeing on their screen to a second user at the second terminal (the “viewer”) because with screen sharing the viewer can see images that are displayed on the sharer's screen.
When the image at the sharer terminal is changed then those changes are transmitted to the viewer terminal, and the image displayed on the viewer's screen can be updated accordingly to reflect the changes. When only certain areas of the image are changed at the sharer terminal then screen rectangles representing those areas in need of updating are transmitted from the sharer terminal to the viewer terminal. In this way it is not necessary to update the whole of the image when only certain areas of the image are in need of updating.
An example protocol for use in transmitting the changes of an image between a server terminal and a client terminal in a Virtual Network Computing system is the RFB (remote framebuffer) protocol (as described in “The RFB Protocol” by Tristan Richardson, RealVNC Ltd, Version 3.8). The display side of the protocol is based around a single graphics primitive: “put a rectangle of pixel data at a given x,y position”. A picture element, or “pixel”, and is the smallest unit of the image that can be controlled. Different encoding schemes for encoding the pixel data can be used. A sequence of the rectangles forms a framebuffer update which represents a change from one valid framebuffer state to another.
One type of encoding which may be used to encode the rectangles in the RFB protocol is RRE encoding (rise-and-run-length encoding). In RRE encoding a rectangle of pixel data to be encoded is divided into rectangular subregions (“subrectangles”) each of which consists of pixels of a single value. The pixel data for a rectangle is then sent to the client terminal in terms of subrectangles representing sections of the rectangle which have a single pixel value.
Another type of encoding which may be used to encode the rectangles in the RFB protocol is Hextile encoding, which is a variation on the RRE idea. In Hextile encoding each rectangle of pixel data to be encoded is split up into tiles of size 16×16 pixels. Each tile is either encoded as raw pixel data, or as a variation on RRE encoding. The pixel data for a rectangle is then sent to the client terminal in terms of the 16×16 tiles.
Other types of encoding may be used to encode the rectangles in the RFB protocol, such as ZRLE encoding (Zlib Run-Length Encoding) in which zlib data when uncompressed represents tiles of 64×64 pixels (similar to the 16×16 tiles of Hextile encoding) and where ZRLE encoding makes use of compressed pixels which can be just 3 bytes long. Other types of encoding which can be used in the RFB protocol are Raw and CopyRect as would be apparent to a skilled person (and as described in “The RFB Protocol” by Tristan Richardson, RealVNC Ltd, Version 3.8).
It can be desirable for the updating of the image on the viewer's screen to be performed in real-time as the image on the sharer's screen is changed. For instance, this can be desirable when the sharer and the viewer are simultaneously engaged in screen sharing and a communication session such as a call or an instant messaging session.